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INTEGRATED TIMELINE AND 
LOGICALLY-RELATED UST VIEW 

Background of Invention 

[0001] Technical Field: 

[0002] The invention is related to a system and method of customizing a software 
program, in this case a presentation program, to provide a user with a graphical 
representation of the order of elements and their timing. This invention is described 
in terms of creating an animation sequence. The invention could, however, be equally 
well applied to other software applications, such as, for example, Internet web-site 
authoring software, project/program management software, scheduling software 
programs, electronic mail programs, and gaming software. 

|[0003] Background Art: 

J[0004] Animation, the adding of a special visual or sound effect to text or an object, is a 
powerful way to attract and sequence an audience's attention in an electronic 
presentation. It is possible to animate text, graphics, diagrams, charts, and other 
objects to focus the audience on important points, control the flow of information, 
and add interest to a presentation. Animation in a presentation occurs when objects 
or animation effects on these objects in a slide are scheduled by the user to appear in 
a prescribed sequence. When the user then inputs a command for one or more of the 
objects or effects to appear, they appear in a certain order. For example, a user can 
have a text bullet on a slide fly into the slide from the left, one word at a time, or hear 
the sound of applause when a picture is uncovered. Such animation effects include 
having an object spin, fade in, fade out, fly across the screen, split, descend, ascend, 
expand, change color, change text characteristics, or be associated with sound or 
video, to name a few. 
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[0005] Animation is a temporally continuous phenomenon, but is typically programmed 
in terms of a discrete sequence of changes. One of the problems facing an author 
producing an animated presentation is that it can be difficult to know which elements 
in the presentation they have animated, the start and stop times (and hence duration) 
of each animation element, and in which sequence the animation events will play. 

[0006] Some existing software that employs animation shows the animation sequence as 
a simple list. This makes it difficult to reconcile the order in which animation elements 
or events are going to animate, and how long they are going to animate. There is no 
indication as to the timing of each animation event and mental effort is required to 
match the list to what is being shown in the animation editor, where the animation is 
composed for later viewing by an audience. Users have complained that they cannot 
tell which objects on the slides are animated, nor can they determine the sequence 
and timing of animation effects. 

[0007] Some software programs use the concept of a simple timeline in order to allow a 
preview of events in a sequence over time. However, these programs require a user to 
work entirely in the temporal plane to determine the sequence of events, adjusting the 
times of each event manually to create the desired sequence of events. This can be 
very cumbersome and time-consuming since using this method to create a sequence 
of events is extremely labor-intensive. In order to preview a sequence of events the 
user must repeatedly search for the times of interest and then take another action to 
display the events occurring at that time. If the user wishes to modify the sequence or 
timing of events, they must then take additional actions to modify the timing of each 
event to produce the desired result. 

[0008] Therefore, what is needed is a system and method for representing or displaying 
the order and timing of objects and effects in an animation sequence, that makes it 
simple for a user to determine which elements are animated, the order in which they 
are animated, and the animation duration for each animation event. Preferably such a 
system would include a great deal of automation to simplify user tasks. 

Summary of Invention 

[0009] 

The system and method according to the present invention solves the problems 
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associated with creating an ordered appropriately-timed animation sequence. It allows 
a user to easily create and edit an animation sequence. The sequence of animation 
events or effects is shown in an event list, each animation event being capable of 
being associated with an event timeline bar that represents the timeline of each 
animation effect and that is correlated with a universal timeline that provides the 
timeline for the entire animation sequence. The animation list, event timeline bars and 
universal timeline are displayed and manipulated in the editing window of an 
electronic presentation or other software that deals with the scheduling of events. 
Logical relationships between events are used to automate and simplify the animation 
sequence creation. The system and method gives a user instant feedback on the 
sequence of animation events, as well as their timing and duration. It allows the user 
to debug the animation events in real time when using an animation editor or viewer 
to create the animation events. Furthermore, it allows the user to easily create a 
simple sequence of events and then fine-tune it with respect to timing of the events. 
The system and method also includes an improved timeline scroll bar that allows the 
user to select desired times and view events occurring at that time in one motion. 

[001 0] Specifically, the system and method according to the present invention consists of 
a User Interface (Ul) that shows the order of the animation events or effects in an 
ordered event list. Each item in the event list represents an individual animation effect 
or event. Each event is preferably associated with a number that represents its order 
in a sequence of mouse clicks or other computer input device selection actions or its 
logical relationship to other animation events in the event list 

^^^^ Elements of the aforementioned event list may be logically related to each other. 

These logical relationships are used in assisting a user to build a sequence of events 
(e.g., an animation sequence). Exemplary logical relationships between events (or 
animation effects) include "start with previous" and "start after previous". The "start 
with previous" logical relationship allows two or more animations to start together at 
the same time as a previous event in the event list. Thus, the 'start with previous' 
logical relationship introduces several items at once in the ordered sequence. On the 
other hand, the "start after previous" logical relationship allows a delay before the 
next animation is introduced. That is, the 'start after previous' logical relationship 
causes an event to start after the completion of an immediately previous event or 
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group of events in the event list. Alternately, the user can specify that an event or 
animation effect should begin with "start on click". This 'start on click' logical 
relationship means that the user can pause their presentation until they click their 
mouse (or other computer input device) a specified number of times. This 'start on 
click' relationship, for example, gives the user time to describe what they are showing 
on a slide. In one embodiment of the invention, the logical relationship can be defined 
to start an event at the same time as another triggering event ('start on trigger'), 
producing an event-driven result. For instance, the user can use this feature to specify 
that a slide containing background information appear if the user clicks on a 
particular portion of a slide. 

[001 2] The benefit for the user of the logical relationships is that the user does not have 
to worry about the timing of the animation eventse.g., the user does not have to 
specify the exact duration of an event or start time in seconds. Nor does the user have 
to manipulate a complex timeline Ul to define a simple sequence of events. 

[001 3] The user may associate a logical relationship with an event in the event list by 
selecting the event and activating a context-sensitive menu that allows the user to 
specify the selected event's logical relationship to other events in the event list. 

[0014] Each event in the event list can also be associated and marked with one or more 
indicators. Such an indicator can, for instance, represent the number in a sequence of 
mouse or other computer Input device selection actions which causes an animation 
effect on an associated object or element. Or the indicator can represent the logical 
relationship of an animation event associated with the object to other animation 
events in the event list. For instance, an indicator that appears as a clock may be used 
to represent the 'start after previous' logical relationship of an event. Alternately, an 
indicator can represent the type of animation effect (e.g., fly in or fade in). 

^1 In one embodiment of the present invention, if the user expands the size of the 
task pane containing the event list in the animation editor then the amount of space 
to show the event list and associated logical relationship indicators is recalculated. As 
the user expands the size of this task pane (and hence gives more space to show the 
information), additional portions of the Ul are displayed (e.g., more indicators or more 
of the event list are displayed). If the user reduces the size of the task pane, less 
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information will be displayed (e.g., the indicators will no longer be displayed, or less 
events will be displayed on the event list). 

[001 6] To allow the user to set more complex timing relationships, an advanced timeline 
view is optionally provided In conjunction with the event list. In general, this detailed 
timeline view graphically shows the timing relationships between events. More 
specifically, each event in the event list may be associated with an event timeline bar 
and correlated to a universal timeline showing times (preferably in seconds or 
portions thereof) that correlate to the event timeline bars displayed in the event list. 
These event timeline bars are graphically depicted to the right of the elements in the 
event list, graphically showing the event's start time and end time. These event start 
and end times correlate to the appropriate times on the universal timeline. Typically, 
only the event timeline bars corresponding to a specific number of mouse clicks are 
displayed (as a group) at a given time. However, it is possible to display all of the 
event timeline bars there is room to display on the slide. The universal timeline scrolls 
in unison with the visible event timeline bars when a user scrolls from left to right (or 
vise-versa) to make different event timeline bars visible. 

[001 7] In one embodiment of the invention, the user can place a computer data input 
device cursor, such as a mouse cursor, over an event timeline bar and a window of 
additional information will pop up. This pop-up window is typically referred to as a 
"tool tip". For example, if the user holds down the left mouse button over the left edge 
on an event timeline bar, the start time of the event will be numerically displayed in a 
pop-up window. If the user holds down the left mouse button over the middle of the 
event timeline bar, the event duration will numerically be displayed. Likewise, if the 
user holds down the left mouse button over the right edge of the event timeline bar, 
the event end time will be displayed. Alternately, this tool tip or pop-up window may 
provide a list of the elements associated with the animation event bar. Or the tool tip 
window could be interactive and allow the user to set up parameters for the animation 
associated with a specific object. 

[001 8] 

The user can 'drag and drop' the event timeline bars (with a mouse or other 
computer input device) to set exact timings. If a users selects and drags the left edge 
of an event bar with an input device the start time of this event will be moved either 
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earlier or later in time, depending on the direction (left or right) the left edge is 
dragged. Assuming horizontally-depicted event timeline bars, if a user selects and 
drags a right edge of an event timeline bar this will move the end time of the event 
either earlier or later in time. Logically, the duration of the event can be modified by 
modifying the start and stop times in this manner. Finally, if a user selects and drags 
the entire event bar with an input device, this action will move the entire event either 
later or earlier in time, while maintaining its duration. 

[001 9] In one embodiment of the present invention, the user can also specify that an 

event be defined as a delay between events. In this case, the delay may be depicted as 
a line from the end of an event timeline bar to the start of the subsequent event 
timeline bar, instead of a bar. Or the event timeline bar may be depicted as a bar that 
is not filled in or that is colored differently from the rest. Additionally, options can be 
made available to the user to insert both long and short standard delay times between 
events. 

[0020] The user of the system and method according to the present invention can use 
various mouse and keyboard short cut keys to manipulate the event timeline bars of 
the event list. For example, 'shift' and 'ctrF keyboard keys can be used to make 
contiguous and non-contiguous multiple selections in conjunction with a mouse. 
Furthermore, the user can change the order of animation effects and associated event 
timeline bars by selecting an animation effect in the animation event list and dragging 
it up or down the list with a mouse cursor or other computer input device. When the 
user changes the order of the animation events by doing this, the logical relationships 
associated with the events in the animation list and the indicators associated with 
them stay the samethe meaning of the logical relationship and indicators attached to 
an effect or event is preserved. The user can also delete an event from the event list 
and the corresponding event timeline bar and associated indicators will be deleted 
and the logical relationships of the other events in the event list will be recalculated 
according to a set of logic rules to reflect their new order in the sequence. 

[0021] 

In one embodiment of the present invention the user can specify that an event or 
effect repeats itself. Typically, this feature is selectable by double-clicking on the 
event from the event list, which brings up a dialog that allows the user to set an event 
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to repeat. Alternately, the user can click the right mouse button to bring up a context- 
sensitive menu that provides the user with an option to set a repeat interval. The user 
can also change the created repeat interval by manipulating the event timeline bars. 

[0022] A single animated object may have zero, one or several event timeline bars 
associated with it in the animation creation and editing portion of the application 
software, each event timeline bar corresponding to a different animation event. 
However, when the final animation product, such as an electronic presentation, is 
viewed by an audience the animation event timeline bars, event list and associated 
universal timeline, are not visible. 

[0023] The present invention's simultaneous showing of the event timeline bar view in 
association with the event list view provides a useful Ul for both novice and expert. 
The list view provides an Ul that makes it easy for a novice to rapidly build a 
presentation sequence. The event timeline bar representation of the animation events 
allows the sequence to then be finessed if the user wants to set more complex 
relationships, and is also useful for performing fine tuning of the timing of events, as 
well as finessing more complex timing relationships such as synchronizing text to 
audio or video. Because the views are combined and allow the user to simply overlay 
the event list and event timeline bars and associated universal timeline within the 
same task pane, the user does not have to do work to understand how the timing of 
various animation events relates to their sequence or vice-versa. The user can switch 
back and forth between working with the event list or working with the event timeline 
bars, depending on whether they want to manipulate exact timing, or just build a 
general sequence. The event list view makes it easy to quickly build a presentation. 
The user can then manipulate and finesse the general sequence of events by dragging 
and dropping the event timeline bars with a computer input device, such as a mouse. 

[0024] 

In one embodiment, when the user is playing an animation, the timeline view is 
shown to make it easy for the user to follow the sequence. The timeline scrollbar is 
shown moving forward in time and the animation events occur as the current time 
marker, the scrollbar's thumb, reaches them. The user is allowed to pause the 
animation playback and directly manipulate the event timeline bars by dragging their 
start or end times. The user can check the event bar timings by 'scrubbing' the thumb 
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of the current time marker (moving within the universal timeline to locate a particular 
time). This process allows the user to get quick feedback as to the current 
arrangement of events. 

[0025] Because the event list view and event timeline bar view are combined, the two 
views allow the user to set timing constraints. These constraints can be shown as 
vertical lines in the timeline view. The setting of timing constraints impacts the user in 
two valuable ways. First, if the user changes the logical relationships with the event 
timeline bar view visible then the event list is re-drawn to show the new sequence. 
This saves the user having to perform cumbersome, time-consuming drag operations 
on the event timeline bars to make them start at the same time, or after another. If the 
user changes the animation event sequence, the event timeline bar is redrawn to show 
the new relationships. Without this feature the user would have to use the mouse to 
align the two event timeline bars, which could be annoying and awkward. Second, if 
the user sets that an effect or event should finish "after" the previous effect then this 
constraint is enforced in the event timeline bar. This means that if the user changes 
another event timeline bar to occur earlier in the sequence then the following event 
timeline bars will be adjusted to accommodate the constraint. This saves the user 
having to manually retime each of the other event timeline bars in the sequence as the 
event timing is automatically reset. 

[0026] Providing flexible constraints in the creation of an event timeline is highly 

powerful. It means that the user does not need to individually retime each timeline if 
they make an adjustment earlier in their presentation the constraint will appropriately 
push all the other event times (and associated event timeline bars) forward or 
backward in time. 

[0027] One embodiment of the present invention contains a specific set of constraint 
satisfaction rules. These are defined as follows:i)The first event timeline bar in the 
sequence is never constrained. It cannot be set to start "after previous", since no 
previous event bar/timeline exists. 

[0028] iDEvent timeline bars set to start "with previous" are not constrained to one 

another. The "with previous" logical relationship is treated as a non-constraint, but 
when it is set the corresponding event is forced to default to start at the start time of 
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the previous event timeline bar. 



[0029] iii)Event timeline bars set to start "with previous" cannot be retimed prior to an 
"after previous". The user is not allowed to drag a "with previous" event timeline bar 
prior to the start of an "after previous". Event timeline bars preceding an "after 
previous" must start either at the same start time, or later. 

[0030] iv)Event timeline bars set to start "after previous" start following the longest 
duration event timeline bar of any "with previous" event timeline bars in the prior 
block or group (usually a group is defined as corresponding to a specific mouse click 
or input device action), up to the last "after previous" event timeline bar. 

[0031] v)Event timeline bars which have no duration i.e., continue forever (such as, for 
example repeating events), are treated as having an end time the same as their start 
time. Setting an "after previous" in this case sets the constraint to the start of the 
timeline. 

[0032] In one embodiment of the present invention, the timeline view contains a 

simplified all-in-one scroll bar and seeking control. This simplified timeline control 
allows the user to seek time and manipulate the view displayed all in one action. As 
the user scrolls forward in time by manipulating a thumb control (a part of the 
scrollbar the user can grab with a mouse cursor or other input device) on a universal 
timeline scrollbar more of the event sequence is made visible to the user. This 
configuration takes up less screen real estate that conventional timeline controls, and 
is more discoverable, since it appears as a normal scrollbar. For comparison of the 
benefits, existing timelines use a separate control to seek time and a normal scrollbar 
to manipulate the view. These can be unwieldy, since the user has to often scroll and 
then use a different control to seek to an exact point in time. The timeline scrollbar 
and associated thumb control of the system and method according to the present 
invention is a unique Ul that makes seeking timing easy and intuitive. 

Brief Description of Drawings 

[0033] 

The specific features, aspects, and advantages of the present invention will 
become better understood with regard to the following description, appended claims 
and accompanying drawings where:FIG. 1 is a diagram depicting a general purpose 
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computing device constituting an exemplary system for implementing the present 
invention. 

[0034] FIGs. 2A and 2B are screen images showing an exemplary implementation of a 

working example according to the present invention wherein the logical relationships 
between events are set in an event list. 

[00351 FIGs. 3A and 3B are screen images showing an exemplary implementation of a 

working example according to the present invention wherein the logical relationships 
between animation events are changed using a context-sensitive menu. 

[0036] FIGs. 4A and 4B are screen images showing an exemplary implementation of a 

working example according to the present invention wherein the task pane displaying 
the event list is widened, 

[0037] FIGs. 5A and 5B are screen images showing an exemplary implementation of a 
working example of the present invention wherein animation events are shown as 
event timeline bars that correlate to a universal timeline. 

[0038] FIGs. 5C is a screen image showing an exemplary implementation of a working 
example of the present invention wherein a pop-up window provides additional 
information on an event timeline bar. 

[0039] FIGs. 5D and 5E are screen images showing an exemplary implementation of a 
working example according to the present invention wherein event timeline bars are 
dragged and dropped with a computer input device to set the exact timing of events. 

[0040] FIGs. 5F, 5G, 5H, 51, 5J and 5K are screen images showing an exemplary 

implementation of a working example of the present invention wherein the event 
timeline bars are dynamically updated when the logical relationship between events is 
changed. 

[0041 ] FIGs. 6A and 6B are screen images showing an exemplary implementation of a 

working example of the present invention wherein the user plays an animation event 
sequence in the animation editor. 

[0042] 

FIGs. 6C and 6D are additional screen images showing an exemplary 
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implementation of the present invention wherein the event bars and timeline are 
dynamically updated when the logical relationship between events is changed. 

[0043] FIGs. 7A and 7B are additional screen images showing an exemplary 

implementation of the present invention wherein a simplified scroll bar is depicted. 

[0044] 

Detailed Description 

[0045] In the following description of the preferred embodiments of the present 

invention, reference is made to the accompanying drawings, which form a part hereof, 
and which is shown by way of illustration of specific embodiments in which the 
invention may be practiced. It is understood that other embodiments may be utilized 
and structural changes may be made without departing from the scope of the present 
invention. 

[0046] Exemplary Operating Environment 

[0047] FIG. 1 illustrates an example of a suitable computing system environment 100 on 
which the invention may be implemented. The computing system environment 100 is 
only one example of a suitable computing environment and is not intended to suggest 
any limitation as to the scope of use or functionality of the invention. Neither should 
the computing environment 1 00 be interpreted as having any dependency or 
requirement relating to any one or combination of components illustrated in the 
exemplary operating environment 1 00. 

[0048] The invention is operational with numerous other general purpose or special 

purpose computing system environments or configurations. Examples of well known 
computing systems, environments, and/or configurations that may be suitable for use 
with the invention include, but are not limited to, personal computers, server 
computers, hand-held or laptop devices, multiprocessor systems, microprocessor- 
based systems, set top boxes, programmable consumer electronics, network PCs, 
minicomputers, mainframe computers, distributed computing environments that 
include any of the above systems or devices, and the like. 

[0049] -ru . . 

The mvention may be described in the general context of computer-executable 
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instructions, sucli as program modules, being executed by a computer. Generally, 
program modules include routines, programs, objects, components, data structures, 
etc. that perform particular tasks or implement particular abstract data types. The 
invention may also be practiced in distributed computing environments where tasks 
are performed by remote processing devices that are linked through a 
communications network. In a distributed computing environment, program modules 
may be located in both local and remote computer storage media including memory 
storage devices. 

[0050] With reference to FIG. 1 , an exemplary system for implementing the invention 
includes a general purpose computing device in the form of a computer 1 1 0. 
Components of computer 1 10 may include, but are not limited to, a processing unit 
1 20, a system memory 1 30, and a system bus 1 2 1 that couples various system 
components including the system memory to the processing unit 1 20. The system bus 
121 may be any of several types of bus structures including a memory bus or memory 
controller, a peripheral bus, and a local bus using any of a variety of bus architectures. 
By way of example, and not limitation, such architectures include Industry Standard 
Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, 
Video Electronics Standards Association (VESA) local bus, and Peripheral Component 
Interconnect (PCI) bus also known as Mezzanine bus. 

[005 1 ] Computer 1 1 0 typically includes a variety of computer readable media. Computer 
readable media can be any available media that can be accessed by computer 11 0 and 
includes both volatile and nonvolatile media, removable and non-removable media. By 
way of example, and not limitation, computer readable media may comprise computer 
storage media and communication media. Computer storage media includes both 
volatile and nonvolatile, removable and non-removable media implemented in any 
method or technology for storage of information such as computer readable 
instructions, data structures, program modules or other data. Computer storage 
media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other 
memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk 
storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic 
storage devices, or any other medium which can be used to store the desired 
information and which can be accessed by computer 1 1 0. Communication media 
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typically embodies computer readable instructions, data structures, program modules 
or other data in a modulated data signal such as a carrier wave or other transport 
mechanism and includes any information delivery media. The term "modulated data 
signal" means a signal that has one or more of its characteristics set or changed in 
such a manner as to encode information in the signal. By way of example, and not 
limitation, communication media includes wired media such as a wired network or 
direct-wired connection, and wireless media such as acoustic, RF, infrared and other 
wireless media. Combinations of the any of the above should also be included within 
the scope of computer readable media. 

[0052] The system memory 1 30 includes computer storage media in the form of volatile 
and/or nonvolatile memory such as read only memory (ROM) 131 and random access 
memory (RAM) 1 32. A basic input/output system 1 33 (BIOS), containing the basic 
routines that help to transfer Information between elements within computer 110, 
such as during start-up, is typically stored in ROM 131. RAM 1 32 typically contains 
data and/or program modules that are immediately accessible to and/or presently 
being operated on by processing unit 1 20. By way of example, and not limitation, FIG. 
1 illustrates operating system 134, application programs 135, other program modules 
1 36, and program data 1 37. 

[0053] The computer 1 1 0 may also include other removable/non-removable, 

volatile/nonvolatile computer storage media. By way of example only, FIG. 1 illustrates 
a hard disk drive 141 that reads from or writes to non-removable, nonvolatile 
magnetic media, a magnetic disk drive 1 51 that reads from or writes to a removable, 
nonvolatile magnetic disk 1 52, and an optical disk drive 1 55 that reads from or writes 
to a removable, nonvolatile optical disk 1 56 such as a CD ROM or other optical media. 
Other removable/non-removable, volatile/nonvolatile computer storage media that 
can be used in the exemplary operating environment include, but are not limited to, 
magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, 
solid state RAM, solid state ROM, and the like. The hard disk drive 141 is typically 
connected to the system bus 121 through an non-removable memory interface such 
as interface 140, and magnetic disk drive 151 and optical disk drive 155 are typically 
connected to the system bus 1 21 by a removable memory interface, such as interface 
150, 
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The drives and their associated computer storage media discussed above and 
illustrated in FIG. 1, provide storage of computer readable instructions, data 
structures, program modules and other data for the computer 1 1 0. In FIG. 1 , for 
example, hard disk drive 141 is illustrated as storing operating system 144, 
application programs 145, other program modules 146, and program data 147. Note 
that these components can either be the same as or different from operating system 
134, application programs 135, other program modules 136, and program data 137. 
Operating system 144, application programs 145, other program modules 146, and 
program data 147 are given different numbers here to illustrate that, at a minimum, 
they are different copies. A user may enter commands and information into the 
computer 1 1 0 through input devices such as a keyboard 1 62 and pointing device 1 61 , 
commonly referred to as a mouse, trackball or touch pad. Other input devices (not 
shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the 
like. These and other input devices are often connected to the processing unit 120 
through a user input interface 1 60 that is coupled to the system bus 121, but may be 
connected by other interface and bus structures, such as a parallel port, game port or 
a universal serial bus (USB). A monitor 1 91 or other type of display device is also 
connected to the system bus 121 via an interface, such as a video interface 1 90. In 
addition to the monitor, computers may also include other peripheral output devices 
such as speakers 1 97 and printer 1 96, which may be connected through an output 
peripheral interface 1 95. Of particular significance to the present invention, a camera 
1 63 (such as a digital/electronic still or video camera, or film/photographic scanner) 
capable of capturing a sequence of images 1 64 can also be included as an input 
device to the personal computer 110. Further, while just one camera is depicted, 
multiple cameras could be included as an input device to the personal computer 1 10. 
The images 1 64 from the one or more cameras are input into the computer 1 1 0 via an 
appropriate camera interface 165. This interface 165 is connected to the system bus 
121 , thereby allowing the images to be routed to and stored in the RAM 1 32, or one 
of the other data storage devices associated with the computer 1 10. However, it is 
noted that image data can be input into the computer 1 1 0 from any of the 
aforementioned computer-readable media as well, without requiring the use of the 
camera 1 63. 
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[0055] The computer 110 may operate in a networked environment using logical 

connections to one or more remote computers, such as a remote computer 1 80. The 
remote computer 1 80 may be a personal computer, a server, a router, a network PC, a 
peer device or other common network node, and typically includes many or all of the 
elements described above relative to the computer 1 1 0, although only a memory 
storage device 181 has been illustrated in FIG, 1. The logical connections depicted in 
FIG. 1 include a local area network (LAN) 1 71 and a wide area network (WAN) 1 73, but 
may also include other networks. Such networking environments are commonplace in 
offices, enterprise-wide computer networks, intranets and the Internet. 

[0056] When used in a LAN networking environment, the computer 11 0 is connected to 
the LAN 1 71 through a network interface or adapter 1 70. When used in a WAN 
networking environment, the computer 1 10 typically includes a modem 1 72 or other 
means for establishing communications over the WAN 1 73, such as the Internet. The 
modem 1 72, which may be internal or external, may be connected to the system bus 
121 via the user input interface 1 60, or other appropriate mechanism. In a networked 
environment, program modules depicted relative to the computer 1 1 0, or portions 
thereof, may be stored in the remote memory storage device. By way of example, and 
not limitation, FIG. 1 illustrates remote application programs 1 85 as residing on 
memory device 1 81 . It will be appreciated that the network connections shown are 
exemplary and other means of establishing a communications link between the 
computers may be used. 

[0057] The exemplary operating environment having now been discussed, the remaining 
parts of this description section will be devoted to a description of the program 
modules embodying the invention. 

[0058] System Overview 

[0059] 1^ ^i^g system and method according to the present invention, a sequence of 
events is graphically shown in an event list, each event being capable of being 
associated with an event timeline bar and correlated with a universal timeline that 
depicts timing for the entire animation sequence. The event list and optimally the 
event timeline bars and universal timeline are typically displayed in the editing window 
of an electronic presentation or other software that deals with the scheduling of 
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events. In one embodinnent of the present invention, each item in the event list 
represents an individual animation effect. The invention gives a user instant feedback 
on the sequence of animation events and their timing and duration during creation of 
the sequence. It allows the user to debug the animation events in real time when 
using an animation editor or viewer. The invention can also include an improved 
timeline scroll bar that allows the user to select desired times and view events 
occurring at that time in one motion. 

[0060] More particularly, the invention consists of a User Interface (Ul) that shows the 

order of the animation events in the presentation in an ordered event list. This event 
list 202 is shown in FIG. 2A. Each item in the list represents an individual animation 
effect or event. Each effect is preferably associated with a number that represents its 
order in a sequence of mouse clicks or other computer input device selection actions 
or its logical relationship to other animation events in the event list. Each event can 
optionally be associated with an event timeline bar and correlated with a universal 
timeline. Elements of the event list are logically related to each other (the logical 
relationships typically being user-specified) and these logical relationships may be 
used in assisting a user to build a sequence of events (e.g., an animation sequence). 

[0061] The following paragraphs describe the system and method according to the 
present invention more specifically. 

[0062] Logical Relationships Between Events 

[0063] Elements of the event list may be logically related to each other. These logical 

relationships are typically set by the user on creation of the animation sequence and 
are used in automating any modifications that are made to the sequence of events. 
Exemplary logical relationships between events (or animation effects) include "start 
with previous" and "start after previous". The "start with previous" logical relationship 
allows two or more animations to start together at the same time as a previous event 
in the event list. Thus, the 'start with previous' logical relationship introduces several 
items at once in the ordered sequence. On the other hand, the "start after previous" 
logical relationship allows a delay before the next animation is introduced. That is, the 
'start after previous' logical relationship causes an event to start after the completion 
of an immediately previous event or group of events in the event list. Alternately, the 
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user can specify that an event or animation effect should begin with "start on click". 
This 'start on click' logical relationship means that the user can pause their 
presentation until they click their mouse (or other computer input device) a specified 
number of times. This 'start on click' relationship, for example, gives the user time to 
describe what they are showing on a slide. In another embodiment of the invention, a 
logical relationship can be defined to start an event at the same time as another 
triggering event (e.g., "start on trigger"), producing an event-driven result. For 
instance, the user can use this feature to specify that a slide containing background 
information appear if the user clicks on a particular portion of a slide. An example of a 
'start on trigger' Ul is shown in FIG. 2B. A divider 210 is shown that lists the name of 
the triggering event. Beneath the divider 210 are listed any events 212 that will play 
when the triggering event is satisfied (e.g., if 'My Button' is clicked on, 'My Text' will 
fly in). 

[0064] The user may associate a logical relationship with an event in the event list by 
selecting the event and activating a context-sensitive menu. This context-sensitive 
menu 302, shown in FIG. 3A, allows the user to specify the selected event's logical 
relationship to other events in the event list. 

[0065] Each event in the event list can also be associated and marked with one or more 
indicators. Such an indicator can, for instance, represent the number in a sequence of 
mouse or other computer input device selection actions which causes an animation 
effect on an associated object or element. Or the indicator can represent the logical 
relationship of an animation event associated with the object to other animation 
events in the event list. In FIG. 38, the clock indicator 304 represents that the 'bullet' 
event will occur after the 'title' event appears. 

[0066] 

In one embodiment of the invention, shown in FIGs. 4A and 4B, if the user 
expands the size of the task pane 402 that contains the event list in the editor, then 
the amount of space to show the event list and associated logical relationship 
indicators is recalculated. If more space is available more of the animation sequence 
will be shown. As the user expands the size of the task pane 402 (and hence gives 
more space to show the information), additional portions of the Ul are displayed (e.g., 
more indicators or more of the event list). If the user reduces the size of the task pane 
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402, less information will be displayed (e.g., the indicators will no longer be 
displayed, or less events will be displayed on the event list), FIG. 4A shows the task 
pane 402 before it is widened. No logical relationship indicators are shown. As can be 
seen from FIG. 4B, the widening of the task pane 402 causes the 'start on click' 
Indicators 404a, 404b, 404c to be shown. 

[0067] Event Timeline Bars and Associated Universal Timeline 

[0068] In another embodiment of the present invention, to allow the user to set more 

complex timing relationships, an advanced event timeline view is optionally provided 
in conjunction with the event list. This timeline view may be selected by a user from a 
context-sensitive menu 500 associated with an event (e.g., 'show advanced timeline' 
502, shown in FIG. 5A). In general, this detailed timeline view graphically shows the 
timing relationships between events. More specifically, referring to FIG. 5B, each event 
in the event list may be associated with an event timeline bar (504, 506, 508) and 
correlated to a universal timeline 510 showing times (preferably in seconds or 
portions thereof) that correlate to the event timeline bars displayed in the event list. 
These event timeline bars are graphically depicted as starting at an event start time 
and ending at an event end time. Typically, only the event timeline bars corresponding 
to a specific number of mouse clicks is displayed (as a group) at a given time. For 
example, if a series of animation events are to occur on the third mouse click, the 
system will display all of the event timeline bars associated with events that will occur 
on the third mouse click, but will not display event timeline bars associated with 
events occurring on the first and second mouse clicks. However, it is possible to 
display all of the event timeline bars there is room to display on the slide or task 
pane. 

[0069] 

In an alternate embodiment of the invention, shown in FIG. 5C, the user can place 
a computer data input device cursor, such as a mouse cursor, over the animation 
event timeline bar and a window of additional information will pop up. This pop-up 
window 511 is typically referred to as a "tool tip". For example, assuming 
horizontally-oriented event timeline bars, if the user holds down the left mouse 
button over the left edge on an event timeline bar, the start time of the event will 
numerically be displayed. If the user holds down the left mouse button over the 
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middle of the event timeline bar, the event duration will be numerically displayed in a 
pop-up window. Likewise, if the user holds down the left mouse button over the right 
edge of the event timeline bar, the event end time will be displayed. Alternately, this 
tool tip or pop-up window 5 1 1 may provide a list of the elements associated with the 
animation event bar. Or the tool tip window could be interactive and allow the user to 
set up parameters for the animation associated with a specific object. 

[0070] The user can drag and drop these event timeline bars (with a mouse or other 
computer input device) to set exact timings. Again, assuming horizontal event 
timeline bars, if a users selects and drags the left edge of an event timeline bar with 
an input device the start time of this event will be moved either earlier or later in time, 
depending on the direction (left or right) the left edge is dragged. If a user selects and 
drags a right edge of an event bar this will move the end time of the event either 
earlier or later in time. Logically, the duration of the event can be modified by 
modifying the start and stop times in this manner. Finally, if a user selects and drags 
the entire event bar with an input device, this action will move the entire event either 
later or earlier in time, while maintaining its duration. An example of the Ul for the 
drag and drop operation is depicted in FIGs. 5D and 5E. FIG. 5D shows the timing of 
the Title event before it is dragged to a later time. FIG. 5E depicts the Title event after 
the event has been moved to occur a second later in time. 

[0071 ] In another embodiment of the present invention, the user can also specify that an 
event be defined as a delay between events. In this case, the delay may be depicted as 
a line, instead of a bar. This line extends from the end of one event timeline bar to the 
start of the next event timeline bar. Or the event timeline bar may be depicted as a 
bar that is not filled in or that is colored differently from the rest. Additionally, options 
can be made available to the user to insert both long and short standard delay times 
between events. 

[0072] 

The user of the system and method according to the present invention can use 
various mouse and keyboard short cut keys to manipulate the event timeline bars of 
the event list. For example, 'shift' and 'ctrl' keyboard keys can be used to make 
contiguous and non-contiguous multiple selections in conjunction with a mouse. 
Furthermore, the user can change the order of animation effects and associated event 
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bars by selecting an animation effect in the animation event list and dragging it up or 
down the list with a mouse cursor or other computer input device. When the user 
changes the order of the animation events by doing this, the logical relationships 
associated with the events in the animation list and the indicators associated with 
them stay the samethat is, the meaning they attach to an event or effect is preserved. 
An example of changing a relationship to an 'after previous' from a 'with previous' 
logical relationship for the 'Bulletl' event is shown in FIGs 5F and 5G. FIG. 5F depicts 
the "before change" condition, while FIG. 5G shows the event timeline bars after the 
logical relationship has been changed. As can be seen from these figures, the event 
timeline bars are changed from all occurring at the same time to 'Bulletl ' (and 'Bullet2' 
that occurs with 'Bulletl ') occurring after the Titlel' event has completed. 

[0073] The user can also delete an animation effect from the animation list and the 
corresponding event bar and associated indicators will be deleted and the logical 
relationships of the other animation events in the event list will be recalculated 
according to a set of logic rules to reflect their new animation order in the animation 
sequence. . FIGs. 5H and 51 provide an example of an embodiment wherein an event is 
deleted. FIG. 5H shows the event list before the Bullet! event is deleted. In FIG. 51, 
which depicts the event timelines after deletion of the Bulletl event, the Bullet2 event 
is then retimed to occur 'with previous' based on the logical relationship associated 
with this event, 

[0074] In one embodiment of the present invention the user can specify that an event 

repeat itself. This feature is selectable by double-clicking on the event from the event 
list to bring up a dialog window that allows the user to set the repeat interval. 
Alternately, the user can right click on the event list entry to bring up a context- 
sensitive menu. This context-sensitive menu provides the user an option to set a 
repeat interval. The user can also change the created repeat interval by manipulating 
the event timeline bars. 

[0075] 

An single animated object may have zero, one or several event timeline bars 
associated with it in the animation creation and editing portion of the application 
software, each event timeline bar corresponding to a different animation event. 
However, when the final animation product, such as an electronic presentation, is 
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viewed by an audience the animation event timeline bars, event list and associated 
universal timeline, are not visible, 

[0076] In one embodiment, when the user is playing an animation, the timeline view is 
shown to mal<e it easy for the user to follow the sequence. The universal timeline 
scrollbar is shown moving forward in time and the animation events occur as the 
current time marker, the scrollbar's thumb, reaches them. The user is allowed to 
pause the animation playback and directly manipulate the event timeline bars by 
dragging their start or end times. The user can check the event bar timings by 
'scrubbing' (moving within the universal timeline to locate a particular time) the thumb 
or current time marker. This process allows the user to get qufck feedback as to the 
current arrangement of events, FIGs. 6A and 6B provide examples of the present 
invention when the user is playing back an animation sequence. FIG, 6A shows the 
universal timeline 608 at 0.5 seconds, and only the 'Title' event has been animated at 
this time on the animation playback display pane 61 0. FIG, 6B shows the universal 
timeline 608 at 1 second, at which time the 'Bulletl ' and 'BulletZ' events are also 
animated (e,g., 'fly in' to the slide) on the animation playback display pane 610. 

[0077] Timing Constraints 

[0078] Because the event list view and event timeline bar view are combined, the two 
views allow the user to set timing constraints. If the user changes the logical 
relationships with the event timeline bar view visible then the event list is re-drawn to 
show the new sequence. If the user changes the animation event sequence, the event 
timeline bar is redrawn to show the new relationships. Without this feature the user 
would have to use the mouse to align the two event timeline bars, which could be 
annoying and awkward. If the user sets that an effect should finish "after" the previous 
effect then this constraint is enforced in the event timeline bar. This means that if the 
user changes another event timeline bar to occur earlier in the sequence then the 
events will be adjusted to accommodate the constraints of the following event 
bars/timelines. This saves the user having to manually retime each of the other event 
timeline bars in the sequence, 

[0079] One embodiment of the present invention contains a specific set of constraint 
satisfaction rules. These are defined as follows. 
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[0080] i) The first event timeline bar in a sequence is never constrained. It cannot be set 
to start "after previous", since no previous event bar/timeline exists. 

[0081] ii) Event timeline bars set to start "with previous" are not constrained to one 

another. Having the "with previous" constraint enforced would make the event timeline 
bar feature annoying to use, since retiming any event timeline bar would force all the 
other event timeline bars preceding it to be retimed as well. The "with previous" 
logical relationship is treated as a non-constraint, but when it is set the 
corresponding event is forced to default to start at the start time of the previous event 
timeline bar. 

[0082] iii) Event timeline bars set to start "with previous" cannot be retimed prior to an 
"after previous". The user is not allowed to drag a "with previous" event timeline bar 
prior to the start of an "after previous". This prevents the user from re-ordering the 
sequence out of the logical order event timeline bars preceding an "after previous" 
must start either at the same start time, or later. 

[0083] iv) Event timeline bars set to start "after previous" start following the longest 
duration event timeline bar of any "with previous" event timeline bars in the prior 
block or group (a group is usually defined as corresponding to a specific mouse click 
or input device action), up to the last "after previous" event timeline bar. This is 
because the "with previous" event timeline bars are not constrained. Hence, it is 
necessary to force an "after previous" to start following the longest duration. 

[0084] v) Event timeline bars which have no duration i.e., continue forever (such as, for 
example repeating events), are treated as having an end time the same as their start 
time. Setting an "after previous" in this case sets the constraint to the start of the 
timeline. 

[0085] Simplified All-ln-One Scroll Bar and Timeline Thumb 

[0086] In oi^e embodiment of the present invention, shown in FIGs. 7A and 7B, the 
timeline view contains a simplified all-in-one scroll bar and seeking control. This 
simplified timeline control allows the user to seek time and manipulate the view 
displayed all in one action. As the user scrolls forward in time by manipulating a 
thumb control on a universal timeline scrollbar more of the event sequence is made 



Page 22 of 44 



visible to the user. (A thumb is the part of a scrollbar the user can grab with his 
mouse cursor or other input device to navigate using the scrollbar.) Additionally, as 
the user holds his cursor to the right edge of the universal scroll bar the events 
displayed will continue infinitely in time. Similarly, if the user holds his cursor to the 
right edge of the scroll bar the events displayed will continue backwards in time 
infinitely. This simplified all-in-one scroll bar takes up less screen real estate that 
conventional timeline controls, and is more discoverable, since it appears as a normal 
scrollbar. For comparison of the benefits, existing timelines use a separate control to 
seek time and a normal scrollbar to manipulate the view. These can be unwieldy, since 
the user has to often scroll and then use a different control to seek to an exact point 
in time. 

[0087] Working Examples: 

[0088] The following paragraphs and figures provide additional working examples of the 
system and process according to the present invention. 

[0089] As shown in FIG. 2A, each event (204, 206, 208) in the presentation to which the 
user has applied animation is shown in the event list 202. The event list 202 is a list 
of the effects which are to play in the slide. This event list 202 can be thought of as a 
script it is read from top to bottom and gives the ordering of each effect to play in a 
slide show. On the left side of the list, a sequence number is provided (204a, 206a, 
208a). The number gives the sequence order in which the animation will play. In this 
example, this number also corresponds to the number of mouse clicks or other input 
device actions that will cause an animation effect to take place (i.e., this is how many 
mouse clicks in the presentation are required to play to this point in the sequence). In 
the example in FIG. 2A, "title" will fly into the slide on the first click, followed by 
"bullet! "on the next mouse click and "bullet2"on another mouse click. 

[0090] 1^ ^f^g system and method according to the present invention, the user can also 
set up using logical relationships between effects to "start with previous" 306 and 
"start after previous" 308, as is shown in FIG. 3A. The principle in this case is that 
"start with previous" allows two or more animations to start together at the same time 
introducing several items at once; whereas "start after previous" allows a delay before 
the next animation is introduced. "Start on click" means that the user can pause their 
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presentation until they click their mouse giving them time to describe what they are 
showing. This user interface makes setting timing and sequencing easy for all users. 
The example in FIG. 3B shows that the Title! ' event is set to animate on the first 
mouse click, then after it is finished animating, the 'Bulletl ' event will animate. On the 
second mouse click the 'BulletZ' event will animate. The user does not have to worry 
about timing e.g., specifying exact duration or start times in seconds, or manipulating 
a complex timeline Ul to define their sequence. 

[0091] Because the timeline and event list views are combined and allow the user to 

simply overlay the two within the same task pane, the user does not have to do work 
to understand how the timeline relates to their sequence or vice-versa; and can switch 
back and forth between the timeline and event list views depending on whether they 
want to manipulate exact timing, or just build a general sequence. This benefit is 
shown in FIGs. 5A and SB. As shown in FIG. 5A, the list view makes it easy to quickly 
build a presentation without worrying about timing or making complex manipulations. 
The user simply creates each animation event and sets its logical relationship using 
the context-sensitive menu 500. The user can then access the timeline view by 
selecting 'show advance timeline' 502 from the menu 500. As shown in FIG. 5B, 
switching to the timeline view shows graphically that the Ttitlel ' event 504 starts on a 
click, and that the 'Bullet!' 506 and 'Buliet2' events 508 will be animated after it, 
because their event timeline bars are placed further along the time axis. The user can 
then manipulate and finesse these event timelines by dragging and dropping the 
events in their entirety, or just the edges of the events to change the start and stop 
times of the events, 

[0092] If the user changes the logical relationships with the timeline view visible then the 
event timeline bars are re-drawn to show the new sequence. This saves the user 
having to perform cumbersome and time-consuming drag operations on the timelines 
to make them start at the same time, or after another. In FIGs. 5J and 5K, the user 
changes the 'Bullet2' event 514 to start after the 'Bullet!' event 512, by using the 
context-sensitive menu 516. The event timeline bars (518, 520, 522) in FIG. 5J are 
redrawn in FIG. 5K (524, 526, 528) to show the new relationships. Without this feature 
the user would have to use the mouse to align the event timelines, which could be 
annoying and awkward. 
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[0093] If the user sets that an effect should finish "after" the previous effect then this 
constraint is enforced in the event timeline. This means that if the user changes 
another event timeline bar earlier in the sequence then all of the following event 
timeline bars will be adjusted. In the example in FIG. 6C, the user retimes the end of 
the "Titlel" event. As shown in FIG. 6D, this also forces the start times of the 'Bullet!' 
and 'BulletZ' events to be changed to maintain the "after" relationship. These 
constraints are shown by the vertical line 602 running from the end of the "Title" event 
timeline bar to the start of the "Bulletl " and "BulletZ" event timeline bars. The 
enforcement of these constraints saves the user from having to manually retime each 
of the other event timeline bars in the sequence. 

[0094] Providing flexible constraints in a timeline is unique and highly powerful. It means 
that the user does not need to individually retime each event timeline if they make an 
adjustment earlier in their presentation the constraint will push all the other event 
timelines forward or backward in time. 

[0095] The combined scrollbar and seeking control of the present invention is shown in 
FIGs. 7A and 7B. The thumb control 704 is used to control the time on the universal 
timeline scroll bar 702. As the user scrolls forward in time by manipulating the thumb 
control 704 more of the event sequence is made visible to the user. Note that the 
constraint between events is shown as a vertical line 706. 

[0096] The foregoing description of the invention has been presented for the purposes of 
illustration and description. It is not intended to be exhaustive or to limit the 
invention to the precise form disclosed. Many modifications and variations are 
possible in light of the above teaching. It is intended that the scope of the invention 
be limited not by this detailed description, but rather by the claims appended hereto. 
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